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Abstract 

Many real incidents demonstrate that users of Online Social Networks need mechanisms that 
help them manage their interactions by increasing the awareness of the different contexts 
that coexist in Online Social Networks and preventing them from exchanging inappropriate 
information in those contexts or disseminating sensitive information from some contexts to 
others. Contextual integrity is a privacy theory that conceptualises the appropriateness of 
information sharing based on the contexts in which this information is to be shared. Com¬ 
putational models of Contextual Integrity assume the existence of well-dehned contexts, 
in which individuals enact pre-dehned roles and information sharing is governed by an ex¬ 
plicit set of norms. However, contexts in Online Social Networks are known to be implicit, 
unknown a priori and ever changing; users relationships are constantly evolving; and the 
information sharing norms are implicit. This makes current Contextual Integrity models 
not suitable for Online Social Networks. 

In this paper, we propose the hrst computational model of Implicit Contextual Integrity, 
presenting an information model for Implicit Contextual Integrity as well as a so-called 
Information Assistant Agent that uses the information model to learn implicit contexts, 
relationships and the information sharing norms in order to help users avoid inappropriate 
information exchanges and undesired information disseminations. Through an experimental 
evaluation, we validate the properties of the model proposed. In particular. Information 
Assistant Agents are shown to: (i) infer the information sharing norms even if a small 
proportion of the users follow the norms and in presence of malicious users; (ii) help reduce 
the exchange of inappropriate information and the dissemination of sensitive information 
with only a partial view of the system and the information received and sent by their users; 
and (iii) minimise the burden to the users in terms of raising unnecessary alerts. 
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1. Introduction 

Online Social Networks (OSNs) have been a source of privacy concerns and issues since 
their early days |3S] . These privacy concerns have even increased along the past decade due 
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to many real privacy incidents being echoed in the media and users being more aware of 
potential privacy issues usiEni. Yet there is a lack of effective privacy controls that allow 
users to satisfactorily manage their privacy in OSNs [69]. In particular, the exchange of 
inappropriate information and the undesired dissemination of sensitive information across 
OSNs are very common and represent one of the major concerns for users. These inappropri¬ 
ate exchanges and undesired disseminations have not only caused serious privacy incidents 
— e.g., users have lost their jobs [SS], have been outed and faced threats to sever family 
ties EDI. have ended their marriages EH, etc. — but also facilitated other activities such as 
social phishing BDl. identity theft [6|, cyberstalking 071, and cyberbullying [63] . 

Some voices argue that this is mainly due to the fact that users are no longer able to 
share information differently for different contexts or spheres of life (friends, work, etc.) 
in the cyber world, as they would usually do in the physical world eg. There are many 
examples in which this is of crucial importance: photos that depict users in embarrassing 
situations, indecorous comments, events that reveal some political affiliations, etc. In all 
these examples, the specific context determines whether or not the exchange of information 
is appropriate — e.g., one may be willing to share her political affiliations with friends but 
not with workmates. 

Contextual integrity [52] is a modern privacy theory that conceptualises the appropriate¬ 
ness of information sharing based on the contexts in which this information is to be shared. 
In particular, contexts are defined considering a set of individuals playing particular roles 
and a number of norms that govern information sharing among them. Contextual integrity 
is said to be maintained — meaning that there are no privacy breaches — whenever these 
information sharing norms are upheld. The information sharing norms have two main pur¬ 
poses: (i) determine what information is appropriate to mention in a particular context, 
and (ii) dictate what information can be transmitted from one party to another or others 
according to the roles enacted by these parties within and across different contexts. 

Computational models of contextual integrity have been recently proposed in the related 
literature [ass]. Following contextual integrity theory, they assume the existence of well- 
defined contexts, in which individuals enact pre-defined roles and information sharing is 
governed by an explicit set of norms. However, contexts in OSNs are “implicit, ever changing 
and not a priori-known” [19]. In particular, the information sharing norms are known to be 
implicit in OSNs [SHI EH], he., they define the behaviour that is consistent with the most 
common behaviour. Moreover, roles are dynamic and may not be known a priori — i.e., 
relationships among individuals in OSNs are constantly evolving m- All of these reasons 
make explicit contextual integrity and the computational models based on it not suitable 
for OSNs. 

In this paper, we present the first computational model of implicit Contextual Integrity 
for OSNs, which includes an information model and an agent model for a kind of agents 
we call Information Assistant Agents (lAAs). lAAs are capable of learning contexts and 
their associated information sharing norms even if these are implicit or unknown a priori. 
Each lAA monitors the information exchanges of its user and based on this it infers: (i) the 
different contexts in which information sharing is to happen; (ii) the relationships among the 
individuals in each context; and (iii) the information sharing norms of each context. If lAAs 
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detect a potential violation of the information sharing norms, they alert their users, who 
have the last word on whether sharing the information or not. Through an experimental 
evaluation we demonstrate that lAAs are able to signihcantly mitigate the exchange of 
inappropriate information and the undesired dissemination of sensitive information while 
minimising the number of unnecessary alerts raised. 

This paper is organised as follows. Section states the problem we tackle in this pa¬ 
per detailing the exchange of inappropriate information and the undesired dissemination of 
sensitive information in OSNs. Section proposes the information model that is used by 
lAAs to promote contextual integrity by alerting users when their actions may lead to the 
violation of the information sharing norms. The lAA model and its modules are described 
in Section Section illustrates the performance of lAAs in a set of examples. Section 
details the experiments we conducted to evaluate our proposal and the results we obtained. 
Section discusses related work. Finally, Section includes some concluding remarks as 
well as future work. 

2. Problem Statement 

lAAs aim to avoid two main privacy threats in OSNs: the exchange of inappropriate 
information, and the undesired dissemination of previously exchanged information. Specih- 
cally, they alert their users that sharing some information may lead to some of these threats. 
Next, we describe these threats in more detail. 

2.1. Inappropriate Information Exchanges 

Each context has its own appropriateness norms that determine which information can 
be mentioned inside each context. For example, one may not mention her political views in 
a work context, but she may do so in a family context |7H|. lAAs use the information that 
their users exchange with other users in one context to infer the appropriateness norms of 
this context. Specihcally, the information that is frequently mentioned by the members of a 
context is considered as appropriate whereas information that is never or rarely mentioned 
is considered as inappropriate. For instance, if most people do not mention their political 
views at work, it could be inferred this is not an appropriate topic to exchange in a work 
context. 

Besides the appropriateness norms of each context, there are situations in which people 
decide to exchange information that may be seen as inappropriate. One of the main reasons 
that explain this is the creation and reciprocation of close relationships [M] . Indeed, there are 
empirical studies that demonstrate the fact that reciprocated communication is the dominant 
form of interaction in OSNs [I5]. Accordingly, lAAs take into account the appropriateness 
of the information that has been exchanged with each user to determine when inappropriate 
information is being exchanged to reciprocate a close friend or to create a close relationship. 

2.2. Undesired Information Disseminations 

Disseminations occur when information disclosed in one context travels to another con¬ 
text. That is, disseminations are inter-context disclosures while exchanges (as stated above) 
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are intra-context disclosures. Obviously, if the information to be disclosed is already known 
in the contexts were it may be disclosed, then the disclosure of this information in these 
contexts cannot entail any new privacy risk. However, disseminations may potentially be 
undesired and hazardous when they entail the disclosure of sensitive information that was 
previously unknown in a context [68]. Indeed, studies on regrets associated to users’ posts 
on OSN highlight the fact that revealing secrets of others is one of the main sources of regret 
|79j . For instance, there was a recent case in which the sexuality of a person was leaked 
from her friends context to her family context where her sexuality was previously unknown, 
causing her being outed and facing threats to sever family ties [30] . 

A hrst-line defence against undesired disseminations may be avoiding sharing sensitive 
information in contexts in which there are people that could disseminate the information 
to other contexts in which this information is previously unknown. Whether these people 
decide to disseminate the information or not may depend on the relationship they have 
to others. That is, people usually have conhdence relationships with others with whom 
they decide to share sensitive information expecting them to keep it secret. One can share 
some of her deepest secrets with her husband but this may not mean her husband would 
disseminate this information to other contexts. Thus, lAAs take into account the knowledge 
of the information that has been exchanged with each user to determine when sensitive 
information is being exchanged to reciprocate a trusted friend or to create/maintain trust 
relationships. 

3. Information Model for Implicit Contextual Integrity 

In this section, we describe the information model for implicit contextual integrity, which 
is used by an lAA (further detailed in Section]^ to assist its user, denoted by a, in sharing 
information in the OSN. 

The information model considers a set of users U = ...,Mn} who are members of 

and communicate in an OSN. Information exchanged among users is formalised as messages 
denoted by tuples {g, R, T, S'); where g eU is the sender user, R GU is the set formed by 
the receiver users, and T and S represent the information that is sent. In general, messages 
represent any exchange of information in an OSN, where this information may be photos, 
comments, private messages, and so on. Let T = be a set of key topics or 

categories that can be discussed in messages. Then, T OR is the set of topics discussed in 
a message, and SOU are the users tagged (i.e., associated) in this message. Informally, we 
dehne a context in our problem as a group of users who interact through a specihc OSN. 
Formally, a context C is a set of users (C OU). We denote by C = {Ci,..., Ci) the set of 
contexts in which user a interacts with other users. We denote by Cp the set of contexts 
that a and fd share: 

= {QIVQ eC-.^eC,} 


3.1. Appropriateness 

As aforementioned, we propose that lAAs infer the information sharing norms from the 
information that is exchanged in the OSN. This section includes the main dehnitions used by 
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lAAs when inferring the appropriateness norms. In particular, we start by dehning the indi¬ 
vidual appropriateness values; i.e., users’ judgement of what is appropriate or inappropriate. 
Then, we define context appropriateness values; i.e., contexts’ appropriateness judgement. 
Finally, we define the appropriateness of messages. 

Definition 1 (Individual Appropriateness). Given a user /3 G W and a topic t ^ T, 
we define the individual appropriateness as a real value within the [0,1] representing the 
likelihood that user fi approves to exchange information about topic t. 


These likelihoods are estimated as frequentist probabilities according to the information 
contained in messages that a receives from other users. If an lAA is created to support a 
user that is new to the OSN, then there is not prior knowledge that the lAAs can use to 
infer the appropriateness of topics. In absence of prior knowledge, the lAA cannot determine 
if exchanging information about one topic is appropriate or not and, as a consequence, a 
initialises the likelihood values to 0.5, which means that it is equally probable that something 
is appropriate or inappropriate [6T1 |39l |20] . If an lAA is created to support a user that has 
been a member of a OSN for a while, then the individual appropriateness values can be 
initialised based on the messages sent by each user fi (i.e., messages sent by /? to a can be 
used to infer to what extent fi approves to exchange information about the different topics). 

Later, the lAA updates these probabilities as it receives messages. For example, if Alice 
(denoted by user a) receives a message from Bob (denoted by user (3) in which a topic such 
as a dirty joke (denoted by t) is mentioned, then this may suggest that Bob considers that 
making dirty jokes is appropriate and, therefore, the likelihood is increased accordingly. 


Definition 2 (Appropriateness Increase). Given an individual appropriateness value 
afp, we define the appropriateness increase as follows: 


f Appropriateness!increase 


mm{l, -\- A(a^)} 


where A : [0,1] —)■ [0,1] is an update function. 


Information sharing norms of a context may change as time goes by; e.g., revealing personal 
information about couples may be appropriate for groups of young friends but it may be¬ 
come inappropriate when these friends grow up and they are engaged in stable relationships. 
In absence of information that sustains the approval of exchanging information about some 
topic, it seems reasonable to consider that it is becoming less usual and finally even dis¬ 
approved. Therefore, the likelihoods that represent appropriateness also decay with time. 


Definition 3 (Appropriateness Decay). Given an individual appropriateness value 
we define the appropriateness decay as follows: 


fAppropriateness Decay g) 


max{0,a^^ - v(a^)} 


where v : [0,1] —t [0,1] is an update function. 
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If /3 exchanges information about one topic, it means that it approves to exchange infor¬ 
mation about this topic. However, if (3 does not exchange information about this topic, it 
does not necessarily mean [3 disapproves to exchange information about this topic. As a 
consequence, we dehne that for any likelihood value 5 G [0,1], A(5) 3> v('^)- 

Given a context G G C and a set of topics T C T, we denote by the set of the 
likelihoods that users in C approve to exchange information about topics in T: 

= {a^l/3 G G A t G T} 

In the information model, the likelihoods that other users approve to exchange informa¬ 
tion about topics are used to determine when a communicative action (i.e., a message sent by 
its user) can be seen as inappropriate. Assume that a wants to send a message ((o;, R, T, S)) 
in which it exchanges information about a set of topics T. Assume also that the message 
belongs to just one context G G C. Thus, a uses its likelihoods that other users in context G 
approve to exchange information about topics in T to determine the appropriateness of this 
message. For example, Alice wants to send a message to Bob in which she makes a dirty 
joke. Given that Bob is a workmate of Alice (the work context is denoted by Cwork), the 
lAA uses the likelihoods that other workmates approve to exchange dirty jokes to determine 
the appropriateness of this message. 


Definition 4 (Context Appropriateness). Given a context C & C and a set of topics 
T CT, we define the context appropriateness as follows: 


fContext Appropriatenessi^C^T') fCorabinei^Affij 


where fcombine ■ [0, —)■ [0,1] is a function that combines the different likelihood values into 

a single value. 


fcombine may be dehned differently according to the requirements of each user. For 
example, extremely cautious users may be interested in dehning it as the minimum to avoid 
mentioning any topic that may be seen as inappropriate by someone. 

Let us assume now that a wants to send a message but that the context of this message 
is unknown a priory. We determine the context of this message considering both the set of 
receiver users and the set of users associated with the information. Informally, we dehne the 
context of a message as the context in which the overlap among its members and the users 
involved in the message (i.e., the receiver and the tagged users) is maximal. For example, 
Alice wants to send a message (denoted by (a, i?, T, S)) to Mary (denoted by user 7 so that 
R = { 7 }) in which she mentions that she. Bob and Gharlie (denoted by user tt), who is also 
a workmate, will attend a training event (thus S = {Q;,/d,7r}). Mary is both Alice’s best 
friend and a workmate; thus, this message might belong to the work and friend contexts. 
Given that the majority of the people involved in the message belong in the work context, 
it is highly possible that this message belongs in the work context. 

Definition 5 (Message Context). Given a message {a, R,T, S) , we define the contexts 
of a message as follows: 


fcontext{{a, R, T,S)) = argmax |G fl (i? U A)] 

cec 
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In our example, Alice wants to send a message to Mary in which she mentions that she, 
Bob and Charlie will attend a training event. In this case, the set of contexts in which Alice 
interacts with other persons are: the work context denoted by Cy/ork (note that /9, tt, 7 G 
Cwork)] and the friend context denoted by Cpriend (note that 7 G Cpriend and (3,7i ^ Crriend)- 
According to de£nition|^ the context of this message is Cwork since \Cworkf^{(3, tt, a, 7 }! =3 
is greater than \CFriend n {/3, tt, a, 7 }! = 1 . 

Note, however, that the previous function may also return more than one context in 
other situations. For example, Alice sends a message to Mary who is a workmate and a 
friend (remember 7 G Cpriend and 7 G Cy/ork) and this message may belong to the work 
and the friend contexts. Thus, any function that calculates the appropriateness of messages 
must take into account that a message may belong to more than one context and that this 
message can be interpreted differently in each context. lAAs want to prevent their users 
from exchanging information about topics that could be seen as inappropriate in any of 
the possible contexts. Thus, they calculate the appropriateness of messages considering the 
context in which this will be less appropriate. 

Definition 6 (Message Appropriateness). Given a message {a, R,T, S), we define the 
message appropriateness as follows: 


fAppropriateness(^(,Oiy RyTj G)) miu fContext Appropriateness (^C^T) 

^^fContext({a,R,T,S)) 


As previously mentioned, people might be willing to exchange inappropriate information 
with those ones that have previously exchanged inappropriate information with them. For 
example, Alice may be willing to exchange a dirty joke with Bob (remember that Bob is a 
work mate), even if this may be seen as inappropriate in the work context because Bob and 
Alice have a close relationship and it is common that they exchange this kind of jokes. To 
take this into account, the lAA needs to know which information has been exchanged with 
the rest of users. We dehne the appropriateness exchange list with (3 (denoted by Ag) as 
the set formed by the appropriateness of all messages that have been exchanged between a 
and f3. Note that this list contains the appropriateness of messages (not the messages) and, 
as consequence, the lAA does not store messages sent or received by its user. In absence 
of prior knowledge, this list is initialised as the empty list. This appropriateness exchange 
list is used by the lAA to know with whom inappropriate information has been exchanged 
in the past and, on the basis of that information, to determine which users have a close 
relationship with its user. 


3.2. Information Dissemination 

Disseminations occur when the information that has been exchanged in one context is 
leaked in contexts where this information is unknown. Thus, it is crucial to enable lAAs 
with capabilities that allow them to anticipate dissemination before it happens. We propose 
that lAAs use the messages sent and received in which users are tagged explicitly (e.g., a 
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user tagged in a photo) to calculate the knowledge of the associations between users and 
topics (e.g., the relationship between Bob and the topic homosexuality) and determining 
which ones are unknown. Again, we start by dehning the individual knowledge values, then 
we dehne context knowledge values, and hnally we dehne the knowledge of messages. 

Definition 7 (Individnal Knowledge). Given any two different users 'y,/3 E U and a 
topie t E T, we define the individual knowledge as a real value within the [0,1] interval 
representing the likelihood that (3 knows the assoeiation of user 7 with topic t. 

As in case of the individual appropriateness values, the individual knowledge values can be 
initialised considering the existence of previous messages received by a. In the absence of 
this information, the individual knowledge values are initialised to 0.5. 

As a receives messages from other users, the lAA uses the information contained in mes¬ 
sages for updating these likelihoods. For example, if Alice receives a message {{/3, R,T, S)) 
in which Bob {/3) tells to Charlie and Alice {R = {Q;, 7 r}) that Mary [S = { 7 }) has been 
promoted (T = {t}; where t denotes the promotion topic), then the lAA has some evidence 
that sustains the fact that Bob knows the association of Mary with the promotion topic. 
Moreover, the lAA also infers that the rest of receiver users (Charlie) know the association 
of Mary with the promotion topic. However, it may be the case that Bob forwarded this 
message without paying much attention to it or analysing it properly. Similarly, it may be 
the case that Charlie never reads this message. Because of this, the lAA cannot be com¬ 
pletely sure that users on S and R know the association of 7 with topic t and it cannot set 
knowledge likelihoods to 1. Thus, the lAA can only increase the likelihood that users on S 
and R know the association of 7 with topic t. 

Definition 8 (Knowledge Increase). Given an individual knowledge value kj’^, we define 
the knowledge increase as follows: 

f Knowledgeincreaseifip ) min\\^ k^ -\~ 3\{kp ) j- 

The information that we know may change as time goes by (e.g., the relationship status of 
our Facebook friends changes along time) or, even, users may forget information (e.g., we 
can forget about the hundreds of Facebook events to which we have been invited), thus the 
knowledge likelihoods decrease lightly with time. 

Definition 9 (Knowledge Decay). Given an individual knowledge value kffi^, we define 
the knowledge decay as follows: 

fKnowledgeDecayikp ) maxfO, ki^ )}’ 

Given a context C E C, user ^ eU and a set of topics T C T we denote by KfF the set 
of likelihood that the association of user 7 with topics in T is known by users in C\ 

= {kf\f3 eC ^tET} 
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Assume that a wants to send a message {{a, R,T, S)) to (3 {R = {/9}) in which user 
7 (S' = { 7 }) is associated with topics T. Upon receiving the message (3 would know that 
association and could disclose it in other contexts. This may cause that f3 disseminates the 
information received from a. This dissemination can be critical if (3 reveals the association 
of 7 with topics in T in contexts where 7 belongs and this association is unknown. Assume 
that 7 and (3 belong to a context C, then a can calculate the knowledge within context C of 
the relationship between 7 and topics in T by considering the likelihood that the association 
of user 7 with topics T is unknown by users in C. For example, Alice knows that Mary is 
pregnant. No one else has talked about Mary’s pregnancy at work. Thus, Alice assumes 
that this information about Mary is unknown at work. 


Definition 10 (Context Knowledge). Given a context C E C, we define the knowledge 
within context C of the relationship between 7 G W and topics in T CR as follows: 


fContextKnowledgeif~3^fcombinei^^Q ) 


Given that any message can be sent to a set of receiver users R and may tag a set of 
users S', the previous function should be extended to several receiver and tagged users. This 
also entails that there may be several contexts where information can be disseminated. The 
lAA wants to avoid disseminations of information that can be unknown in some context. 
Thus, the lAA calculates the knowledge considering the least known information that can 
be disseminated. For example, Alice may want to send a message to her family (S') in which 
she tells about Mary’s pregnancy. Charlie is her brother-in-law {71 E R). Charlie also works 
at the same company (i.e., Mary and Charlie share the work context) and, as a consequence, 
Charlie might reveal the information about Mary’s pregnancy at work. Thus, this message 
may entail the dissemination of unknown information at the work context. 

Definition 11 (Message Knowledge). Given a message {a, R, T, S), we define the knowl¬ 
edge of a message as follows: 


fKnowledgei^ip^i R-iT^ Dlin fContextKnowledge{C ^ S 

VC'G(CsnCr) 

where Cg H Cr represents the contexts shared by s and r. 

As previously mentioned, the creation and maintenance of trust relationships explains 
the fact that people might be willing to exchange unknown information which those ones 
who have previously exchanged unknown information with them. The lAA need to know 
the knowledge of the information that has been exchanged with the rest of users. We dehne 
as the knowledge exchange list with (3 (denoted by JCg) as the set formed by the knowledge 
of all messages exchanged between a and f3. In absence of prior knowledge, this list is 
initialised as the empty list. The knowledge exchange list is used by the lAA to determine 
which users have a trust relationship with its usei0 Exchanges of unknown information 


^Note that having a close relationship is not the same as having a trust relationship; thus, we need to 
maintain two independent exchange lists. 
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with these trusted friends would be justihed by the disclosure reciprocity phenomenon. For 
example, the fact that Alice tells Charlie the information about Mary pregnancy (which 
may be disseminated in the work context) would be justihed by the fact that Charlie is a 
trusted friend of Alice. 

4. Information Assistant Agent model 

Figure depicts the different modules that make up an lAA. We can see that it is aimed 
to be placed between one user and one OShQ Thus, an lAA is responsible for managing 
the interactions (i.e., the messages sent and received) of a single user in an OSN. An lAA 
could be implemented on top of an OSN infrastructure (or Social Network Service) as a 
Social Networking App running in a trusted server or it could be implemented as a local 
Social App that acts as a proxy to access the OSN (e.g., a mobile Social App). Note that as 
stated in the previous section, an lAA does not store messages sent/received by their users, 
which helps to minimise storage space for the information model, which may be important 
to implement an lAA as a local App for resource-constrained devices like smartphones. Note 
also that lAAs utilise information that is currently available to users in mainstream OSNs 
and their applications —i.e., the content posted and read by users (e.g., tweets) and the 
friendship relationships (e.g., the following and follow relationships). 

An lAA is composed of 5 different modules: (i) community hnding algorithm; (ii) passing 
of time function; (iii) message sending function; (iv) message reception function; and (v) the 
information model. We already detailed in the previous section the information model, so 
this section focuses on explaining the rest of modules, how the information model interacts 
with them, and a description of an lAA’s life-cycle. 

4 . 1 . Community Finding Algorithm 

In OSNs, the individuals that belong to a particular context are commonly referred to 
as a community [32]. These communities [32] are natural divisions of network users into 
densely connected subgroups. Many OSNs support some notion of community by means of 
allowing groups of users (e.g., Facebook groups or Google-|- circles). There are even methods 
and ready-to-use tools that obtain them automatically using unsupervised machine learning 
algorithms [52] • For example, lAAs could make use of existing community hnding algorithms 
to extract the different communities, such as Infomap [62], which is known to perform 
accurately in OSNs like Facebook [26] and is the algorithm lAAs use for the examples and 
evaluation we carried out, as described later on in the paper. Note that OSN infrastructures 
only allow each user to have a partial view of the OSN (that contains his/her friends and 
mutual friends). That is, each lAA needs to elicit the contexts in which its user is involved 
by considering a partial view of the network (also known as ego-network). 

The community hnding algorithm can be executed by lAAs on the background without 
interfering with the users’ actions. Specihcally, lAAs can recalculate the communities at 
regular time intervals or after an event, e.g., any time there is a change in the friend network. 

^Note that our model does not consider explicitly the existence of multiple interacting OSNs. However, 
multiple interacting OSNs can be combined using the approach proposed in [^ and m- 
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Figure 1: Components and placement of an Information Assistant Agent. 


J^.2. Passing of Time 

The pseudo code corresponding to this function is depicted in Algorithm At this 
step, an lAA updates likelihoods as the times goes by. Specihcally, the lAA applies the 
decay functions to the appropriateness likelihoods to model the fact that, in absence of new 
information that sustains the appropriateness of exchanging information about a topic, this 
appropriateness degrades over time (see line 3). Similarly, the lAA applies the knowledge 
decay function, which models the fact that, in absence of new information that reveals the 
association between a user and a topic, the knowledge of such association decreases (see line 
6 ). 


Algorithm 1 Passing of Time Function 


1 

2 

3 

4 

5 

6 

7 

8 
9 

10 


for all P €U do 
for all t G T do 

i fAppropriatenessDecayi^ajf) 

for all 7 G do 
if 7 / /3 then 

i fKnowledgeDecay{kjj ) 

end if 
end for 
end for 
end for 


Note that the passing of time function can be executed on the background by an lAA at 
regular time intervals without delaying its user’s actions. 

f.3. Message Reception 

The pseudo code corresponding to this function is depicted in Algorithmic At this step, 
an lAA processes the messages received by its user. Messages are received in the input 
mailbox [InBox). The lAA treats the messages received in the input mailbox in FIFO 
order for: 

1. Updating the exchange lists. The lAA computes the appropriateness of all messages 
its user receives to update the appropriateness exchange list with the sender user of 
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each message (line 3). Similarly, the knowledge of messages is used to update the 
knowledge exchange list (line 4). 

2. Updating the appropriateness likelihoods. All topics mentioned in a message are used 
to update the likelihood that the sender approves to exchange information about these 
topics (line 6). 

3. Updating the knowledge likelihoods. All topics exchanged in a message are used to 
update the likelihood that the association of the tagged users with the topics is known 
by the sender (line 8). Similarly, the likelihood that the association of the tagged users 
with the topics is known by the rest of message receivers is increased as well (line 10). 


Algorithm 2 Message Reception Function 


1 

2 

3 

4 

5 

6 

7 

8 
9 

10 

11 

12 

13 

14 


while InBox is not empty do 

Retrieve (/3, R, T, S) from InBox 

jA/j i U AppropriatenessUP: R) Fj 'S)))} 

ICp {fKnowledgeiil^, R-, T, S))} 

for all t G T do 


^ fAppr opiateness I ncreaseiP' 

for all s G S' do 

fKnowedgelncreasei}^p ) 


Kp 

for all r G R do 


rxj'p > 

end for 
end for 
end for 
end while 


fKnowledgeIncrease{k' 


S,t\ 


The message reception function is executed every time a new message is received. In 
the worst case, the temporal cost of this function is 0{TF^), where T is the number of 
topics and F is the number of friends in the social network. Among adult Facebook users 
the median number of friend^ is 200. For example, works on NLP state that there are 36 
key concepts on written English Rayson. According to recent statistics, the average social 
network user receives 285 pieces of content dailjj^ Thus, on average the lAA executes the 
message reception function, which has a polynomial cost, less than 300 times a day. Besides 
that, that this function does not require the intervention of users. That is, the messages 
received by an lAA can be straightforwardly delivered to its user while the lAA executes 
the message reception function on the background. 

4.4- Message Sending 

The pseudo code corresponding to this function is depicted in Algorithmic At this step, 
the lAA processes the messages that its user wants to send before they are actually sent. 


^Source: http://www.pewresearch.org/fact-taiik/2014/02/03/6-new-facts-about-facebook/ 
^Source: http://www.iacpsocialmedia.org/Resources/FunFacts.aspx 
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The user leaves the messages she wants to send in the output mailbox [OutBox). Before 
these messages are sent, the lAA analyses the messages and tries to prevent their users from 
sending messages in which inappropriate information would be exchanged and/or unknown 
information could be disseminated. In particular, the lAAs analyses the messages to be sent 
to: 

1. Check the appropriateness of messages, which is computed for each message (line 4). 
The value of the message appropriateness models the suitability of a message with 
respect to the information sharing norms. Besides, and as aforementioned, people 
usually reveal inappropriate information (e.g., embarrassing information) to create 
and/or to reciprocate close relationships. Thus, the appropriateness of the messages 
that have been exchanged with each receiver is computed as well (line 4). This value 
models the level of closeness of the relationship between the sender and the receiver. 
To compute the appropriateness of the messages that have been previously exchanged 
with each receiver, the lAA uses /Aggregate ■ [0,1]" —t [0,1], which is a function that 
maps a set of real numbers within the [0,1] interval to a real number within the [0,1] 
interval. This function aggregates all the appropriateness likelihoods (vs. knowledge 
likelihoods) stored in the exchange list with a receiver user Ar into a single value. 
After this, the lAA checks the appropriateness of the message being analysed with 
respect to the appropriateness of the messages that have been previously exchanged 
with the receiver. If the appropriateness of a message is less than the appropriateness 
of the messages that have been exchanged with the receiver, then the lAA alerts its 
user about the risks of such action and asks its user to conhrm the sending operationj^ 

2. Avoid undesired disseminations. An lAA also tries to avoid that its user sends messages 
that may lead to the dissemination of sensitive and unknown information. Specihcally, 
the knowledge of messages is also computed for each message (line 11). The value of 
the message knowledge models the sensitivity of a message. The lower the knowl¬ 
edge of a message, the higher the probability it contains sensitive information whose 
dissemination is undesirable. People usually share unknown information to create or 
maintain trust relationships (e.g., I would reveal my husband conhdential information 
about a common friend). The knowledge of the messages that have been exchanged 
with each receiver is also computed (line 11). When the knowledge of a message is 
less than the knowledge of the messages that have been exchanged with the receivers, 
then the lAA alerts its user about the risks of such action and asks its user to conhrm 
the sending operation. 

3. Update the exchange lists. If the message is hnally sent, then the lAA computes 
the appropriateness of the messages sent by the user to update the appropriateness 
exchange list with the receiver users (line 18). Similarly, the knowledge of messages is 
used to update the knowledge exchange list (line 19). 


^For the sake of clarity and simplicity, we only show in this paper where the alerts would be raised, 
but please note that the alerts could be much more informative than what is shown in Algorithm For 
instance, an lAA could use all the information it has about the receivers, information appropriateness, and 
previous messages exchanged to craft a more informative alert to be conveyed to its user. 
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4. Update the knowledge likelihoods. If the message is hnally sent, then all topics men¬ 
tioned in a message are used by the lAA to update the likelihoods that the association 
of the tagged users with topics mentioned in the message is known by the receivers 
(line 22). 


Algorithm 3 Message Sending Function 
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while Out Box is not empty do 

Retrieve {a, R, T, S) from OutBox 

for r G i? do 

if fAppropiateness{{(^T R^T, < fAgreggatei.'^r) then 

continue ■(— receiveDecision{^T\i\s may be inappropiate’) 
if ^continue then return 
end if 
end if 
end for 
for r G i? do 

if fKnowledgeii^j ^)') ^ fAgreggatei^^r) then 

continue •(— receit'eUecision(‘This may disseminate sensitive information’) 
if ^continue then return 
end if 
end if 
end for 
for r G i? do 

Aj- i Aj- U {fAppropiateness{,{oi^ Ri F, S'))} 

/Cj- i /Cj. U {f Knowledge{{c^j A, T, S))} 

for all t G T do 
for all s G S do 


kp^ G 

end for 
end for 
end for 

Send {a, R, T, S) 

end while 


fKnowledgeincreasepk' 


S,t\ 


The message sending function is executed before a new message is actually sent. This is 
the only function that may require the intervention of users by asking them to confirm the 
sending operation. That is, if the lAA raises an alert, its user needs to conhrm whether the 
message should finally be sent or not. Otherwise, the message will be sent straightforwardly. 
In the worst case, the temporal cost of this function is 0{TF‘^), where T is the number 
of topics and F is the number of friends in the social network. Recent studies on sharing 
behaviour on social network, showed that users users send messages less frequently than 
they receive message^ Thus, the message sending function will be executed significantly 
less times than the message reception function. 


®Source: http://www.pewinternet.org/files/2015/01/PI_SocialMediaUpdate20144.pdf 
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As aforementioned, there is a single lAA situated between the user and the ONS. How¬ 
ever, it may be possible to distribute the computation performed by an lAA into a set of 
specialized lAAs. For example, the messages sent and received can be distributed among 
specialized lAAs according to the context. Thus, each lAA is responsible for controlling 
the interactions of a user within a given context. Finally, it is also possible to dynamically 
adapt the number of lAAs by performing cloning and self-deletion operations. However, the 
dehnition of more elaborated procedures for adapting dynamically to changing environments 
lai is a complex issue that is out the scope of this paper. 

5. Illustrative Examples 

This section illustrates the performance of the lAA in a few representative examples. In 
particular, these examples are aimed at showing the capabilities of lAAs to deal with: (i) 
evolving relationships between users; (ii) dynamic contexts; and (iii) distinguishing unusual 
information from inappropriate information. 

5.1. Evolving Relationships 

To illustrate the performance of lAAs when dealing with evolving relationships between 
users, let us assume the existence of a user a who has just started working at a new company. 
a uses the lAAs to interact with his workmates, represented as the context Cwork = {/3,7, 5}, 
through an OSN. Initially, none message has been exchanged between them. As a result, 
all the individual appropriateness values are set to 0.5 (i.e., Vc G C^jorkR G T '■ a\ = 0.5). 
Similarly, the appropriateness exchange lists are empty (i.e., Vc G Cyjork : Vic = 0). From 
that moment on, several messages containing the work topic (i.e., messages about tasks, 
meetings, etc.) are exchanged between them. Thus, the work topic is seen by all users as 
appropriate (i.e., \/c & C : ~ 1), whereas the rest of topics are seen as inappropriate 

(i.e., Vc G CyjorkR & T '. t ^ work A a* ~ 0). Similarly, the appropriateness exchange 
lists contain a list of increasing values (i.e., Vc G C^ork '■ Vic ~ {0.5, 0.7,..., 1,1,1}), which 
represents the fact that the work topic has become more appropriate to the group as they 
have advanced in their professional relationship. 

At some point, a and (3 realised they both support Manchester United and they begin to 
exchange some football videos between them. Obviously this causes that the lAA infers that 
(3 considers as appropriate to exchange information about the sport topic (i.e., ~ 1). 

However, the lAA infers that among the work friends the sport topic is mostly inappropriate 
(i.e., fcontextAppropiateness{Cwork, {sport}) < 1), since the rest of friends never mentioned it. 
This causes that the lAA infers that f3 has become a close colleague and the relationship 
between them has evolved (i.e., Ag = 0.3, 0.3}), since there are some common 

interests that lead to messages including inappropriate topics. Because of that, when a 
sends a message to {3 with the sport topic no alert is raised (note that the lAAs knows 
that the sport topic is inappropriate for the work context, but a and [3 usually exchange 
information that is not appropriate for this context). 

Sometime afterwards, (3 becomes a boss and he thinks that he needs to have a more 
professional relationship with his workmates (the relationship evolves again). As a result, f3 
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(a) Initial Situation (b) First Photography Class (c) Second Photography Class 

Figure 2: Friend Network in the Evolving Context Scenario. Users are represented by 
smiley faces, friend relationships between users are represented by lines, and contexts are 
represented by bold ellipses. 


stops sending messages about the sport topic, and only exchanges work-related information 
with a. As a consequence, the appropriateness exchange list reflects that lately (3 has only 
being exchanging appropriate information (i.e., Ajs = {..., 1,1,1}). Thus, the next time a 
tries to send a message to (3 containing football information, the lAAs will raise an alert 
indicating that this may been seen as inappropriate. Then, a must decide between sharing 
the information anyway (trying to rebuild the relationship) or not sharing, thus abiding by 
the new roles. 


5.2. Evolving Contexts 

In this example, we will focus on a situation in which contexts evolve. Let us assume 
the same user a with the same three workmates (3,'y and 6 . a has not dehned any group 
on the OSN and the lAA uses the community Ending algorithm to infer that f3 ,7 and 6 
belong to the same context (i.e., The social network corresponding to 


this situation is depicted in Figure 2a 


At some point, a and 5 start a weekly photography course. In the first class, they meet 
e and become his friends on the social network. This situation is represented in Figure 
As a result of this change in a’s friend network, the lAA executes the community finding 
algorithm. In the absence of more information about user-generated groups, the algorithm 
assigns e to the work context (i.e., C^ork = 
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In the second week, 6 decides to leave the course whereas a and e attend the second 
class. In that class, they meet ( and r] and become their friends on the social network. This 
situation is represented in Figure As a result of this change in a’s friend network, the 
lAA executes the community hnding algorithm. Thanks to the new friend relationships, the 
algorithm identihes correctly the work context (i.e., Cwork = and the photography 

course context (i.e., Ccourse = {e,C)h})- 

This example illustrates how lAAs are able to deal with the dynamic creation of unfore¬ 
seen contexts. 

5.3. Distinguishing Inappropriate Information from Unusual Information 

As previously mentioned, the likelihood values of the proposed information model are 
estimated as frequentist probabilities. This entails that topics that are frequently discussed 
by users in a context are seen as appropriate. Topics that are never mentioned may be inap¬ 
propriate or unusual. This example illustrates how lAAs distinguish between inappropriate 
and unusual information. 

Let us assume the same users a, /3 ,7 and 6. As previously mentioned, these users use the 
OSN to exchange work-related information. Thus, the topic work is considered as appropri¬ 
ate by the lAAs. Any time a decides to send a message discussing work-related issues, the 
lAA processes the message without raising any alert. At some point, a sends a message con¬ 
taining an invitation to his wedding. The lAA raises an alert informing a about the fact that 
the message may be seen as inappropriate (i.e., f contextAppropiateness{C.^ork, {marriage}) = 0 
since Vc G C^ork ■ = 0). a conhrms the sending operation. As a result, / 3,7 and 6 

express their joy (e.g., they like the original post) and reply to the original post (e.g., they 
make comments on the post indicating whether or not they will attend the wedding). All 
these messages entail that the lAA increases the appropriateness of the marriage topic (i.e., 
Vc G Cwork '■ S> 0). When a decides to send other messages with further details 

about the wedding, the lAA does not raise any alert. 

Let us assume a different situation in which a decides to send a message containing an 
obscene joke. Given that the obscene topic has never been mentioned before, the lAA raises 
an alert (i.e., fcontextAppropiatenessiCwork'j {obsCCnc}) 0 siuce Vc G Cwork • ^)‘ 

However, a decides to send the message anyway. However, /3 ,7 and 5 think that the joke is 
in bad taste and they ignore it (e.g., they do not comment it, like it or retweet it). As a result, 
the appropriateness of the obscene topic does not increase (i.e., Vc G Cwork '■ = 0). 

When a decides to send another message containing an obscene joke the lAA raises an alert. 
At this point, a might realise that their friends do not approve this kind of information and 
decide to stop sending these messages. 

This example shows how lAAs can distinguish between inappropriate and unusual in¬ 
formation. In both cases the lAA raises an alert when the hrst message is sent. However, 
the lAA only continues to raise alerts when the information is inappropriate (e.g., obscene 
jokes), not when it is unusual (e.g., wedding details). In particular, the fact that users of 
OSNs usually show their support for specihc comments, pictures, posts, etc. can be exploited 
to distinguish unusual form inappropriate information. Indeed, recent statistics show that 
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this form of communication (i.e., likes and comments) is prevalent in OSN^ 

6. Evaluation 

To analyse how the lAAs works and to validate our hypothesis about their behaviour, 
this section analyses the properties of lAAs regarding: contextual integrity maintenance, 
information sharing norm learning, alert notifications to their users, effect of environmen¬ 
tal parameters, and robustness to malicious users. To this aim, we conducted a series of 
experiments illustrating the behaviour exhibited by lAAs considering a wide range of dif¬ 
ferent situations and conditions, modelling scenarios with real OSN characteristics (e.g., 
scale-free networks, few strong ties, etc.) and heterogeneous users with different attitudes 
and preferences. 

6.1. Setting Description 

We considered users who are members of an OSN and who initially have some pieces of 
information that can be exchanged. As they receive information from others, they acquire 
new pieces of information that can also be exchanged. Parameters and their default values 
(which change depending on the experiment) are described in Table Each user on the 
OSN has been implemented as a user agent. There is a set U of 100 user agents among 
which friendship relationships are randomly created following a scale-free pattern, according 
to empirical studies that prove OSNs are scale-free [19]. In our scenario, we assume that 
users can only exchange information with their friends (as occurs in Facebook and other 
mainstream OSN). A subset of these friendship relationships is randomly selected to be 
close and trusted friend relationship^ Users are grouped forming contexts or communities. 
The set of contexts (C) in each simulation (considering the complete view of the social 
network) is elicited using the Infomap algorithm [62], which is a clustering algorithm that 
has demonstrated high performance when identifying Facebook contexts [26]. Note that 
these are unknown to lAAs, which only have an approximation to these contexts in each 
simulation by running the same algorithm but with a partial view of the social network from 
their users point of view (as stated in Section]^. This makes even more difficult for lAAs 
to learn the information sharing norms, but this models the conditions lAAs will actually 
face in real OSNs. 

Next, we describe how the information sharing norms, users, and lAAs were modelled: 

6.1.1. Information Sharing Norm Modelling 

At the beginning of each simulation, we created a set of information sharing norms, which 
determined the information that is inappropriate in each context and the information that 
is sensitive (i.e., mostly unknown) in each context and should not be disseminated. Note 
that these norms are implicit and a priori unknown by users and agents. 


^According to Instagram statistics, on average 2.5 billion likes are sent diary, whereas only 70 million 
photos are uploaded per day. Source: https://instagrain.com/press/ 

®For simplicity, in our simulations we have assumed that all close relationships are also trust relationships. 
However, our model is able to infer these two types of relationships independently. 
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Parameter 

Default Value 

number of user agents 

100 

number of topics 

36 

maximum number of topics per message 

36 

maximum ratio of inappropriate topics in each context 

10% 

maximum ratio of sensitive associations in each context 

1% 

number of steps 

2000 

number of executions 

100 


Table 1: Parameters and their default values. 


Definition 12. Formally, the set of information sharing norms is defined as follows: 


Inappropiate d) ffSensitive 


where: 


• Minappropiate is the Set of inappropuateness norms that is formed by {t, c) expressions, 
which represent that talking about topic t G T is seen as inappropriate in context c G C; 

• -dfsensitive is the sct of scnsitivencss norms which is formed by {u, t, c) expressions, 
which represent that the association of user u E U with topic t E T is sensitive (i.e., 
mostly unknown) in context c E C. 

We considered a set T of 36 different topics that represent key concepts (e.g., sports, politics, 
etc.) and that can be recognised in English text [60]. In each context, we dehne randomly the 
topics that are inappropriate and associations between users and topics that are sensitive: at 
maximum 10% of these topics are inappropriate (we vary this later on), so that the number 
of inappropriate topics in each context belongs to the interval [1,4]; and at maximum 1% of 
the possible associations between users and topics are sensitive (we vary this later on), so 
that the number of sensitive associations in each context belongs to the interval [1, 36]. 

Social norms, as information sharing norms in OSN, are not created by legal or insti¬ 
tutional authorities. They are established in societies by means of a bottom-up process in 
which they emerge from individual behaviour [22]. Specihcally, social norms emerge when 
they are followed by a considerable portion of the society. To simulate the existence of the 
information sharing norms (i.e., the appropriateness and sensitiveness norms lAAs need to 
learn), we model a subset of the users in the OSN to be compliant users, who do not use 
lAAs but know the norms and follow them; i.e., they simulate people that have the ability 
to say always the right thing. Thus, compliant users only send messages that respect the 
appropriateness and sensitiveness norms. 

6.1.2. User Modelling 

Apart from compliant users, we modelled the behaviour of different types of users accord¬ 
ing to a classihcation of different normative attitudes based on Westin’s privacy attitudes 

milTSI: 
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• Random users are norm unconcerned — i.e., users who do not care about the in¬ 
formation sharing norms, so they do not use lAAs and randomly compose messages 
containing information that they know and send them to a random subset of their 
friends. These messages can cover any set of topics. 

Note that ours is the first model of implicit contextual integrity, thus we cannot com¬ 
pare the performance of lAAs against previous proposals. As a consequence, we will 
use random users (which model users who do not use our methods to maintain con¬ 
textual integrity) as a baseline method for comparison, which simulates the current 
situation in all OSN infrastructures (like Facebook, Google-1-, etc.). 

• Obedient users are norm fundamentalists — i.e., users who adhere to the information 
sharing norm^ They utilise lAAs and are called obedient since they always follow 
the recommendations made by their lAAs. Similarly to compliant users, they also try 
to comply with the information sharing norms, but they do not know the norms and 
rely on lAAs to infer and abide by the norms. 

• Relationship-based users are norm pragmatists — i.e., users who make use of casual 
forms of information sharing that are not always normative, e.g., when they interact 
with a close or trusted friend. They utilise lAAs but they decide not to follow the 
recommendations made by lAAs when they send messages to close and trusted friends. 

6.1.3. I A As Implementation 

As aforementioned, the information model used by lAAs makes use of several functions 
(i.e., update functions, combination function and aggregation function). The propose of this 
paper is not to provide or compare different definitions to these functions, but to make use of 
these functions in evaluating anew computational model of contextual integrity. The specific 
definitions may depend not only on the application domain but also on the personality 
traits of user^^ As a result, in our experiments lAAs functions are given well-known and 
reasonable definitions. 

In particular, we define A as a constant update rule; i.e., V5 G [0,1] : A(5) = x where 
X = 0.1. Similarly, we dehne v a constant update rule; i.e., V5 G [0,1] : v('5) = V 
where y = 0.01. Constant update rules, which have been previously used in the literature 
on norm learning and emergence assume that each lAA will update its likelihood values 
in constant increments or decrements. 

f Combine is implemented as a harmonic mean, since it mitigates the influence of large 
outliers (e.g., close workmates to which inappropriate information is frequently sent). 

/Aggregate is implemented as a weighted mean where each element is weighted by its index; 
so that the most recent elements contribute more to the final value. Formally: 

c A n,- 

_ ^ 

J Aggregate\^ ) 

l^Vi&S * 


®Note that compliant users can also be classified as norm fundamentalists, 
user study to generate personalised lAAs is beyond the scope of this paper. 
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Finally, lAAs initialize randomly the likelihood that other users approve to exchange 
topics or know the associations of users with topics. This models a situation in which lAAs 
have varied and imperfect prior knowledge availabl^^ More formally, for all user /3 G W and 
topic t eT the value is set to a random real value within the [0,1] interval. Similarly, for 
all pairs of users /?, 7 G W, where 7 <> j3 and topic t E T the value k'^’^ is set to a random 
real value within the [ 0 , 1 ] interval. 

6.2. Experiments 

6.2.1. Contextual Integrity Maintenance 

In our hrst experiment we sought to determine if lAAs contribute to maintain contextual 
integrity, i.e., if lAAs help to reduce the exchange of inappropriate information and the 
dissemination of sensitive information. To this aim, we simulated societies populated with 
compliant users (who know and follow the norms) and obedient users (who use lAAs to infer 
and follow the norms). Each simulation was executed during 2000 steps and was repeated 100 
times to support the findings. For each execution, we calculated the messages that exchange 
inappropriate information and the messages that disseminate sensitive information. We also 
repeated this experiment increasing the percentage of compliant users (who know and follow 
the norms) in each context from 0% to 90% to test lAAs in a wide range of situations. The 
rest of users in each context are obedient users. As a baseline simulation, we repeated this 
experiment using compliant users increasing from 0% to 90% and the rest being random 
users (who do not make use of lAAs). 

Figure shows the behaviour exhibited by random and obedient users in terms of the 
exchange of inappropriate information. This figure depicts the average number of messages 
that exchange inappropriate information sent by all obedient users and by all random users 
per ratio of compliant agents. This hgure shows that obedient users (who make use of lAAs) 
exchange less inappropriate information than random users (who do not use lAAs) regardless 
of the ratio of compliant users. On average, the exchange of inappropriate information in 
societies populated by compliant and obedient users is reduced by 90% compared to societies 
populated by compliant and random users, which clearly shows that lAAs contribute to 
maintain contextual integrity. One can also observe that the exchange of inappropriate 
information in societies populated by random users decreases as the ratio of compliant 
users increases. Obviously, the less random users, the more compliant users and the less 
exchanges of inappropriate information occur. This very same effect can be observed in 
obedient users when the ratio of compliant users belongs to the [40%,90%] interval. As 
the ratio of compliant users increases more appropriate information is exchanged in the 
network and obedient users can infer which topics are appropriate and the exchange of 
inappropriate information decreases more easily. In contrast, when there are few compliant 
users (i.e., when the ratio of compliant users belongs to the [0%,40%] interval) that exchange 


^^Note that if we model situations in which all users are new to the OSN and lAAs have no prior knowledge 
available, which is highly unlikely in practice; this may lead to all lAAs having the same initial values and 
to the emergence of fake norms corresponding to this shared initialization. 
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appropriate information, then obedient users infer that almost all topics are inappropriate 
and they end up exchanging less message^^ 


Number of Messages that Exchange Inappropiate Information (Random Users) 
Number of Messages that Exchange Inappropiate Information (Obedient Users] 



Ratio Compliant Users 


Figure 3: Average number of messages that exchange inappropriate information sent by user 
type (Y-axis) per ratio of compliant users (X-axis) 

Figure compares the behaviour exhibited by random users against obedient users in 
terms of the dissemination of sensitive information. This figure depicts the average percent¬ 
age of messages that disseminate inappropriate information sent by all obedient users and 
by all random users per ratio of compliant agents. As we can observe, the results are very 
similar to the ones above, i.e., obedient users disseminate less sensitive information regard¬ 
less of the ratio of compliant users. On average, the dissemination of sensitive information 
in societies populated by compliant and obedient users is reduced by 89%. 

6.2.2. Information Sharing Norm Learning 

In the previous experiment, we demonstrated that lAAs maintain contextual integrity 
when the network is formed by compliant users (who know and follow the norms) and 
obedient users (who try to learn and follow the norms). A more complex scenario arises 
when lAAs deal with societies in which there are users that do not care about the information 
sharing norms (i.e., random users). In this situation, it is more difficult to infer the norms 
on the basis of the information exchanged, since random users exchange appropriate and 
inappropriate information indistinctly. To assess to what extent lAAs are able to learn 
the information sharing norms in this kind of situations, we performed an experiment in 
which we increased the percentage of compliant users in each context from 0% to 90%. 


^^Note that lAAs are not able to distinguish compliant users and they infer the norms from the information 
that is exchanged in the social network. 
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Number of Messages that Disseminate Sensitive information (Random Users) 



Ratio Compliant Users 

Figure 4: Average percentage of messages that disseminate sensitive information sent by 
user type (Y-axis) per ratio of compliant users (X-axis) 


The rest of users in each context was split in 10% of obedient users and the rest being 
random users. Thus, we sought to determine if lAAs are able to infer the norms when 
they are not predominant in the society. Again, the simulation was executed during 2000 
steps and repeated 100 times. For each execution, we calculated the messages that exchange 
inappropriate information and the messages that disseminate sensitive information. 

Figure [^displays the average percentage of messages sent by obedient users (who make 
use of the lAAs) that exchange inappropriate information and disseminate sensitive infor¬ 
mation per ratio of compliant agents. This hgure shows that the exchange of inappropriate 
information and the dissemination of sensitive information are reduced noticeable as the 
ratio of compliant users increases. For example, when there are 40% of compliant users 
(which implies that there are 10% of obedient and 50% of random users) in each context 
the exchange of inappropriate information is reduced by 35% and the dissemination of sen¬ 
sitive information is reduced by 27%. This demonstrates that lAAs learn the information 
sharing norms even if compliant users are not predominant in contexts. With regard to 
disseminations, we can observe that in general less sensitive disseminations occur. This is 
explained by the fact that sensitivity norms relate a specihc user with a specihc topic and 
context, whereas inappropriateness norms only relate a topic with a context. Thus, sensitive 
information is scarcer than inappropriate information. When the ratio of compliant users is 
very high, then there are more sensitive disseminations than inappropriate exchanges. This 
is explained by the fact that sensitiveness norms are more difficult to infer (since they relate 
a specihc user and topic). As a consequence, the reduction on the exchange of inappropriate 
information as the ratio of compliant users increases is less pronounced. 
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Figure 5: Average percentage of messages sent by obedient users that exchange inappropriate 
information and disseminate sensitive information (Y-axis) per ratio of compliant users (X- 
axis) 


6.2.3. Alerts Raised 

This experiment is aimed at assessing the performance of lAAs in terms of the number 
of alerts raised and how many of these alerts are not followed by users. To this aim, we 
executed simulations populated with compliant users and relationship-based users (who do 
not follow the recommendations of lAAs when they exchange information with close and 
trusted friends). We set the percentage of compliant users in each context to 40% to model 
already emerged information sharing norms, and we increased the number of steps that 
of the simulation to determine the number of steps lAAs require to infer the information 
sharing norms as well as close and trust relationships. Note that learning close and trust 
relationships allow lAAs to avoid raising alerts when violations of the information sharing 
norms are explained because the recipients are close or trusted friends. These alerts may be 
considered as unnecessary and disturbing since users are unlikely to follow them. 

Figure shows that the percentage of messages that raise alerts and the percentage of 
messages that raise non-followed alerts decrease exponentially to a low value as the number 
of steps increases, i.e., the percentage of messages that raise alerts converges to a value below 
25% and the percentage of messages that raise non-followed alerts converges to a value below 
5%. This demonstrates that lAAs require a low number of steps to infer the information 
sharing norms, and trust and close relationships. Also, the low percentage of non-followed 
alerts (5%) demonstrates that lAAs minimise the burden on the users in terms of raising 
alerts that would not be followed. 
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Figure 6: Average percentage of raised alerts and average percentage of non-followed alerts 
(Y-axis) per number of steps (X-axis) 

6.2.4- Simulation Parameters 

In the previous experiments, we fixed the maximum number of topics discussed in a 
message and the maximum ratios of inappropriate topics and sensitive associations. To 
analyse the influence of these parameters on the performance of lAAs, we carried out two 
experiments in which we hxed the percentage of compliant users in each context to 40% to 
model already emerged information sharing norms and the rest of users in each context are 
obedient users. We repeated these two experiments using random users instead of obedient 
users as a baseline simulation. Each simulation was executed during 2000 steps and repeated 
100 times. 

Message Composition. In our previous experiments, we wanted to assess the performance 
of lAAs in a general situation; i.e., when users are allowed to select any number of topics to 
be discussed in a message. Thus, we did not impose any limit on the size of messages. To 
analyse the impact of the maximum number of topics discussed in the messages (i.e., the 
message size) on the lAAs performance, we carried out an experiment in which we increased 
the maximum number of topics that can be discussed in a message from 1 to 35. 

Figures!^ and [^compare the behaviour exhibited by random users against obedient users 
in terms of the exchange of inappropriate information and the dissemination of sensitive 
information with respect to the maximum number of topics that can be discussed in a 
message. As depicted in these two images, the more the topics discussed in messages, 
the higher the average percentage of messages that violate some information sharing norm. 
Obviously, the more topics are discussed in a message, the higher the probability of including 
an inappropriate or sensitive topic. However, the increase on the average percentage of 
messages that violate some information sharing norm is noticeably higher in random users 
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— i.e., the lines corresponding to random users have a signihcantly higher slope than the 
lines corresponding to obedient users. This demonstrates that even if the users are not able 
to discern norm-compliant topics to be discussed in messages, lAAs would help them do so. 

% of Messages that Exchange Inappropiate Information (Random Users) 

70% I •••••% of Messages that Exchange Inappropiate Information (Obedient Users) 



Figure 7: Average percentage of messages sent by user type that exchange inappropriate 
information (Y-axis) per maximum number of topics (X-axis) 
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Figure 8: Average percentage of messages sent by user type that disseminate sensitive 
information (Y-axis) per maximum number of topics (X-axis) 
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Norm Definition. To analyse the impact of the number of norms on lAAs’ performance, 
we carried out an experiment increasing the maximum ratio of inappropriate topics in any 
context from 5% to 30% and the maximum ratio of sensitive associations between users and 
topics from 0.5% to 3%. 

Figure compares the behaviour exhibited by random users against obedient users in 
terms of the exchange of inappropriate information with respect to the maximum ratio of 


inappropriate topics. Figure 10 compares the behaviour exhibited by random users against 
obedient users in terms of the dissemination of sensitive information with respect to the 
maximum ratio of sensitive associations between users and topics. In case of random users, 
the average percentage of messages that violate some information sharing norm increases as 
the ratio of inappropriate topics or the ratio of sensitive associations increases. When the 
information to be discussed in a message is randomly selected, the probability of exchanging 
sensitive or inappropriate information increases as more information is considered illegal 
according to the information sharing norms. In case of obedient users, we obtained the op¬ 
posite results (i.e., the average percentage of messages that violate some information sharing 
norm decreases slightly as the two ratios increase). Note that the amount of information 
(i.e., information sharing norms) that lAAs have to infer does not depend on the ratio of 
inappropriate topics or sensitive associations (i.e., for each topic lAAs have to infer to what 
degree it is sensitive or not, appropriate or not) and, as a result, the performance of our 
model is not affected negatively by the increase on the value of these ratios. On the con¬ 
trary, lAAs obtain slightly better results when the ratio of inappropriate topics or the ratio 
of sensitive associations increases. This is explained by the fact that if more information 
is considered illegal according to information sharing norms, then compliant users (who al¬ 
ways comply with information sharing norms) have less information to include in messages. 
Because of this, obedient users receive norm-complaint information at a higher frequency, 
which facilitates the inference of information sharing norms. Thus, lAAs’ performance scales 
very well with increasing ratios of information sharing norms. 

6.2.5. Robustness against Malicious Users 

This experiment aimed to analyse the performance of lAAs in presence of malicious 
users; i.e., attackers who intend to inject fake messages so as to increase the appropriateness 
of a topic. Malicious users exchanged messages that included malicious information (i.e., 
information relative to one inappropriate topic targeted by malicious users). In particular, 
we fixed the percentage of obedient users in each context to 10%, and we varied the ratio 
of malicious users from 0% to 90%. The rest of users in each context were compliant users. 
We repeated this experiment with random users instead of obedient users as a baseline 
simulation. Each simulation was executed during 2000 steps and repeated 100 times. 


Figure 11 shows the behaviour exhibited by random users and obedient users in terms 
of the exchange of inappropriate information. The average percentage of messages sent by 
random users that exchange inappropriate information decreases as the percentage of mali¬ 
cious users increases. This is explained by the fact that, malicious users always exchange the 
same malicious information, whereas compliant users exchange any information as long as 
this information is appropriate. However, a piece of information may be seen as appropriate 
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Figure 9: Average percentage of messages sent by user type that exchange inappropriate 
information (Y-axis) per max. ratio of inappropriate topics (X-axis) 
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Figure 10: Average percentage of messages sent by user type that disseminate sensitive 
information (Y-axis) per max. ratio of sensitive associations (X-axis) 

in a given context and as inappropriate in another context. Thus, when compliant users 
are replaced by malicious users, less information is available in the network, so that random 
users have less information at their disposal, which makes them exchange less inappropriate 
information. On the contrary, in obedient users, the average percentage of messages that 
exchange inappropriate information increases as the percentage of malicious users increases. 
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% of Messages that Disseminate Sensitive Information (Random Users) 










However, the average percentage of messages that exchange inappropriate information in¬ 
creases more slowly than malicious users do. In particular, lAAs prove to be robust against a 
single malicious user and small to medium-sized groups of malicious users. Indeed, obedient 
users behave as random users only when they are completely surrounded by malicious users. 

% of Messages that Exchange Inappropiate Information (Random Users) 



Figure 11: Average percentage of messages sent by user type that exchange inappropriate 
information (Y-axis) per ratio malicious users (X-axis) 


6.2.6. Realistic User Populations 

The previous experiments were aimed at assessing the properties of lAAs with regards 
to some specihc criteria. For this reason, we generated societies with extreme distributions 
of user types. However, these settings may not seem realistic; e.g., a situation in which all 
users are unconcerned about the norms is unlikely. In contrast, this section presents a more 
realistic simulation in which the distribution of the different user types match the results 
of empirical studies on privacy concerns among the general public [ 721111 ]. In particular, 
and according to [ 72 |, we have defined the ratio of obedient and compliant users (i.e., norm 
fundamentalists) as 26% — from which 15% are compliant users to model the emergence 
of norms and the rest are obedient users; the ratio of relationship-based users (i.e., norm 
pragmatists) as 64%; and the ratio of random users (i.e., norm unconcerned) as 10%. Besides, 
recent studies demonstrate that in OSNs the majority of relationship ties are weak [2I1IHI|. 
In particular, and according to [HI], we have defined that 10% of the friend relationships are 
close and trusted relationships. Finally, each simulation is executed during 4000 steps and 
has been repeated 100 times. 

Figure [T^ displays the average percentage of messages sent by each type of users that 
exchange inappropriate information per number of steps that the simulation is executed. 
Similarly, Figure IB displays the average percentage of messages sent by each type of users 
that disseminate sensitive information per number of steps that the simulation is executed. 
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When the number of steps is very small, users have received few messages and they have a few 
pieces of information to be sent. Thus, the probability of violating information sharing norms 
is low. As the number of steps increases, users have received more pieces of information 
and they have more options to violate information sharing norms. As illustrated by these 
figures, all users that use lAAs (i.e., obedient and relationship-based) exchange significantly 
less inappropriate information and disseminate significantly less sensitive information than 
random users. Specifically, obedient users are the ones that violate less information sharing 
norms (obedient users achieve a reduction on norm violations over 70%). This demonstrates 
that thanks to lAAs, obedient and relationship-based users, who don’t know the information 
sharing norms, are able to comply with them. 

of Messages that Exchange Inappropiate Information (Random Users) 

• ••••% of Messages that Exchange Inappropiate Information (Obedient Users) 

« M % of Messages that Exchange Inappropiate Information (Relationship-based Users) 



Steps 

Figure 12: Average percentage of messages sent by each user type that exchange inappro¬ 
priate information (Y-axis) per number of steps (X-axis) 


7. Related Work 

7.1. Contextual Integrity Modelling and Reasoning 

Previous work on computational models of contextual integrity proposed mechanisms 
for modelling and reasoning about contextual integrity principles. For example, Barth et al. 
[1] formalized some aspects of contextual integrity assuming that there is a set of explicitly 
defined norms that determine what is permitted and forbidden, that the interactions take 
place in well-known contexts, and that interaction participants play a specihc role in each 
interaction. In a more recent work, Krupa et al. |13] proposed a framework to enforce 
information sharing norms in an electronic institution where norms, contexts and roles 
are explicitly defined. While these approaches seem appropriate for the kind of domains 
described in |3] and [12], in OSNs there are not well-known contexts, there is not an explicit 
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Figure 13: Average percentage of messages sent by each user type that disseminate sensitive 
information (Y-axis) per number of steps (X-axis) 


dehnition of the roles played by users and the exchange of information is governed by implicit 
information sharing normf^ 

7.^. Access Control Models for OSNs 

The suitability of traditional access control models such as role-based access control [M] 
for OSNs has been recently challenged on the basis that they cannot capture the inherent 
social nature of OSNs, such as social relationships and distance among users. To address this 
limitation, there are new paradigms that precisely emphasise the social aspects of OSNs. 
These new paradigms range from cryptographic privacy systems used to dehne and im¬ 
plement group access policies 121 ESI SSI Eg, to relationship-based access control models 
[271 ESI EEl El sa Ea ITS] that utilise a variety of features or aspects to characterise users’ 
relationships and dehne access control decisions based on them. 

While these new models represent better frameworks than other traditional access con¬ 
trol approaches to develop tools for dehning and enforcing access control policies in OSNs, 
access control on its own is unlikely to be the complete and dehnitive solution for an appro¬ 
priate privacy management in OSNs as users need awareness about access control decisions 
to fully understand the consequences of their access control policies miEni Eg. For in¬ 
stance, access control models are known to fail to prevent unintended disclosures [g. This 
is where the implicit contextual integrity approach presented in this paper can make a dif¬ 
ference by complementing these new models by learning implicit information sharing norms 


^•^Note that these implicit information sharing norms are normality norms that define the behaviour that 
is consistent with the most common behaviour. In contrast, explicit information sharing norms define 
behaviour that is normative (i.e., moral). 
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in OSNs and warning users where their defined access control policy may mean that inap¬ 
propriate information would be exchanged or sensitive information would be disseminated 
across OSNs. 

1.3. Disclosure Decision-Making Mechanisms 

In the related literature, the use of software endowed with disclosure decision-making 
mechanisms is not new. For example, several authors [761 113 proposed mechanisms for 
computing the privacy-beneht trade-off of information disclosures in online interactions. 
The aim is to only disclose information when this trade-off renders appropriate results, i.e., 
where the utility of a particular disclosure is worth the privacy risks/consequences involved 
by performing the disclosure. However, these mechanisms have difficulties to deal with sce¬ 
narios where the direct benefit of disclosing a piece of information is a priori unknown or 
difficult to express in economic terms, such as OSNs, in which disclosures are mostly driven 
by social factors [38] • In a more recent work. Such et ah un proposed a mechanism for 
entailing agents with capabilities to select the personal attributes of their users to be dis¬ 
closed to other agents during interactions considering the increase on intimacy and privacy 
loss a disclosure may cause. However, this mechanism does not consider that the appropri¬ 
ateness of disclosures may vary from context to context, nor does it consider information 
disseminations. Also, this mechanism is fully automated whereas in our proposal lAAs only 
warn users when information sharing norms may be violated but users have the last word 
on whether information is hnally shared or not. 

7.4- Topic Analysis 

Our proposal is based on the existence of a mechanism that is able to determine the 
topics discussed by the pieces of information that are exchanged in OSNs (i.e., the messages). 
Topics here can be categories predehned by the underlying OSN infrastructure [5l|; user¬ 
generated tags like Flickr categories [53] ; or categories or tags extracted from images |66ll80]- 
videos [3, geolocation information [13, or text [S71E11II3. 

A well-known problem of user-generated tags is that the tagging terms are often ambigu¬ 
ous, overly personalised and inexact EH. Indeed, current OSNs offer little or no synonym 
(i.e., different words, same meaning) or homonym (i.e., same word, different meanings) con¬ 
trol [331I18] - To overcome these problems, recent advances in the field of information sciences 
have proposed mechanisms for clustering semantically related tags (i.e., referred as topics in 
our paper) [211 ESI El] • 

With regard to topic extraction from text, current research in the held of NLP has made 
advancements on analysing short messages present in OSN, microblogs, etc. For a review 
of these works see |S7|. Specihcally, there are some NLP proposals endowed with new text¬ 
mining and analysis techniques that analyse short and informal messages with acceptable 
accuracy 0. For example, the Relative Forensics toolkilp3 employs, among other features, 
the 36 topics used in our experiments to analyse messages online messages. Specihcally, 
the Relative Forensics toolkit has demonstrated to obtain a successful performance when 
detecting deceptive behaviour in online messages [58] . 

^■^http: //www. relative-forensics . com 
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7.5. Norm Learning 

Norm learning [23] is the process of learning how to behave in a specific sitnation. In 
case of ONS information-sharing norms are implicit (i.e., there is not an explicit definition 
of what is sensitive or inappropriate), and supervised machine learning algorithms cannot 
be used to infer information sharing norms. 

In the existing literature, social learning mi of norms is dehned as the process of inferring 
implicit social norms concurrently over repeated interactions with members of the social 
network. In most of the proposals on social learning, norms are inferred by analysing the 
outcomes of interactions and normative decisions in terms of utility [HS]- As previously 
mentioned, in OSN the beneht of exchanging information may be difficult to be determined 
in economic terms. In other proposals, norms are inferred by analysing explicit normative 
signals such as punishments, sanctions and rewards m These approaches cannot be used 
in OSN since implicit information sharing norms are product of informal social control that 
is rarely stated explicitly (e.g., sanctions) to unfriendly individuals. Other approaches [23], 
use imitation as a mechanism for learning social norms. In these proposals, the norms are 
inferred from the public behaviour exhibited by the majority of the members of the social 
network (or the majority of the members within an observation radius). A main drawback 
of imitation approaches is that all members are equally considered; i.e., they do not consider 
the existence of different social contexts with different social norms and the fact that users 
engage in relationships of different nature and strength. 

These unsupervised machine learning approaches are unsuitable to be applied to ONS. 
This paper goes beyond these imitation approaches by considering the specific characteristics 
of OSNs, i.e., the existence of different and implicit social contexts with different information 
sharing norms, and the existence of different types of relationships between users. In partic¬ 
ular, and to the best of our knowledge, our proposal is the first model of social learning of 
implicit information sharing social norms based on public and private information exchanges 
in OSNs. 

8. Conclusions 

In this paper, we proposed the hrst computational model for implicit contextual integrity 
in OSNs. This model considers that in OSNs contexts are implicit, unknown a priori and 
they may change; user relationships evolve; and information sharing norms are implicit, i.e., 
they define the behaviour that is consistent with the most common behaviour. This model 
of implicit contextual integrity includes an information model for inferring the information 
sharing norms that are in force in OSNs and an Information Assistant Agent model to help 
users avoid exchanging inappropriate information and disseminating sensitive information 
across ONSs. 

The experiments we conducted show that: (i) lAAs are able to infer information sharing 
norms even if a small proportion of the users follow the norms and in presence of malicious 
users; (ii) lAAs help to reduce the exchange of inappropriate information and the dissemi¬ 
nation of sensitive information with only a partial view of the system and the information 
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received and sent by their users; and (iii) lAAs minimise the burden to the users in terms 
of raising unnecessary alerts. 

As future work, we plan to equip lAAs with active mechanisms for detecting malicious 
users. As stated in the experiments, lAAs are robust against a single malicious user and 
small to medium-sized groups of malicious users. However, this robustness may not hold 
when an lAA is completely surrounded by a large number of malicious users. This does not 
mean that the user of the lAA would be more prone to disclose inappropriate information, 
but that in the event she decides to disclose inappropriate information, the lAA may not 
alert her. Besides, note that these large groups of users would hrst need to be friends with 
lAA’s user (or successfully inhltrate en masse in her friend network), share contexts with 
her, and coordinate targeting a particular topic without the user noticing it. Automated 
approaches could be envisioned so that a single entity (whether individual or organisation) 
controls a large number of accounts to launch such attacks, but this can be mitigated using 
existing sybil defences [DEE]. 
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